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Comparing the signals measured by the surface and underground scintillator detectors of the 
Yakutsk Extensive Air Shower Array, we place upper limits on the integral flux and fraction of 
primary cosmic-ray photons with energies E > 10 18 eV, E > 2 x 10 18 eV and E > 4 x 10 18 eV. 
The large collected statistics of the showers measured by large-area muon detectors allows to test 
the photon fraction as small as a few per mil, thus opening the possibility to probe the chemical 
composition of cosmic rays at super-GZK energies, density of their astrophysical sources and rele- 
vant characteristics of the extragalactic medium, as well as to further constrain exotic new-physics 
scenarios. 
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1. Ultra- high-energy (UHE) cosmic-ray photons are 
secondary particles naturally expected to be produced 
by energetic protons and nuclei interacting with various 
backgrounds at acceleration sites or/and along their tra- 
jectories towards the Earth [l[. 

While the observation of the suppression of the cosmic- 
ray flux at energies E >6 • 10 19 eV by the High Resolu- 
tion Fly's Eye (HiRes) [2| and by the Pierre Auger Ob- 
servatory (PAO) Q and lack of the Galactic anisotropy 
at E > 10 19 eV suggest that the most energetic cosmic 
particles have the extragalactic origin, the chemical com- 
position of these particles is still largely uncertain. While 
the HiRes measurements of the mean depth of air-shower 
development 0] and of its fluctuations [j| are fully consis- 
tent with pure proton composition, a similar analysis of 
the PAO data [6| suggests a substantial fraction of heavy 
primaries at E > 10 19 eV. The latter possibility is sup- 
ported by the Yakutsk muon data and by the studies 
of the shower front structure by PAO 0. Therefore, in- 
dependent tests of the chemical composition at extreme 
energies are of high importance. 

One of these tests is provided by a search for primary 
photons with somewhat lower energies, E > 10 18 eV. 
Indeed, both protons and heavier nuclei with energies 
E ~ 10 20 eV experience intense interaction with cos- 
mic background radiations, especially with microwave 
(CMB) and infrared (IRB) photons. The processes in- 
volved in these interactions are however very different. 
Interactions of a proton at E > 7 x 10 19 eV with CMB 
photons lead to efficient pion production through baryon 
resonances. The proton loses about 20% of its energy 
per each 5 Mpc of travel (see, e.g., 3). This energy 
drag implies the Greisen-Zatsepin-Kuzmin (GZK) cut- 
off |9j, [l0( in the cosmic ray spectrum at highest energies. 



Further decays of neutral pions produced in these interac- 
tions provide with the secondary photon flux at energies 
E > 10 18 eV (so-called GZK photons) @, ED]. On the 
other hand, the dominant interaction channel for heav- 
ier nuclei is the photodesintegration on the IRB photons. 
The protons emerged in this reaction may further pho- 
toproduce on CMB; however, their energies are typically 
smaller than the resonance threshold. As a result, both 
the spectral suppression is less pronounced [9l. | ipL ll2l| and 
the secondary photon flux is much smaller [131 ]. There- 
fore, the observation of relatively large photon flux at en- 
ergies E > 10 18 eV would speak in favor of the absence 
of heavy nuclei at highest energies. UHE photon spec- 
trum at the Earth depends both on original spectrum 
at the sources and on details of travelling through the 
Universe. Thus, if measured, photon component gives 
a possibility to probe acceleration mechanism operating 
in the sources, their density and relevant characteristics 
of extragalactic medium (photon background, magnetic 
fields, etc.) 

On the other hand, the study of UHE photons is a 
powerful tool for constraining new-physics models. One 
example is provided by models with superheavy dark- 
matter particles. Originally suggested [lj] to explain the 
high energy cosmic ray events beyond the GZK-cutoff 
mostly due to the AGASA excess [IH [lj|, the models 
per se are not ruled out. The superheavy-particle produc- 
tion in the early Universe is naturally driven by gravita- 
tional expansion at the post-inflationary preheating stage 
[13, EH- This mechanism is quite generic and is capable of 
explaining dark matter, if particles are long-lived enough. 
The superheavy particles can be naturally unstable due 
to either gravity-suppressed couplings [FJ, 1 1911 or non- 
perturbative quantum field theory processes [201 ] . Dark- 
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matter particles are collected in galaxies and their decays 
contribute to the observed cosmic-ray flux. Generally, a 
substantial fraction of the decay products are photons, 
which in the energy range of interest, E > 10 18 eV, 
penetrate straight through the Galaxy. The distribution 
of arrival directions of these photons over the celestial 
sphere is expected to exhibit anisotropics (cf. Ref. [2l|). 
reflecting the dark-matter distribution in the Galaxy and 
the Local Group. Searches for the UHE photons, as well 
as neutrinos, give a unique possibility to hunt for the su- 
perheavy dark matter. One more class of exotic relics to 
be searched for with cosmic rays are topological defects. 
These objects emit highly energetic particles contribut- 
ing to the flux of UHE cosmic rays (see e.g. [H,[23j]) and 
UHE photons were suggested [24| as a signature of this 
process. 

With the help of UHE photons, one may also constrain 
astrophysical models of the cosmic-ray origin which in- 
volve new physics at the propagation stage. In particu- 
lar, both the spectrum and the chemical composition of 
cosmic rays are changed in models with violation of the 
Lorentz invariance [25J. UHE cosmic rays are particles 
with highest Lorentz-boost factors, and hence the very 
place to probe these models. Photon fraction at high- 
est energies is sensitive to parameters violating Lorentz 
invariance, and upper limits on the former severely con- 
strain the latters [26J. Finally, photons with energies 
above ~ 10 18 eV might be responsible for cosmic-ray 
events correlated with BL Lac type objects at the angu- 
lar scale significantly smaller than the expected deflection 
of protons in cosmic magnetic fields and thus suggesting 
neutral primaries [53, [28[ (see Ref. [29| for a particular 
mechanism) . 

In this Letter we present the analysis of extensive air 
showers observed by The Yakutsk Extensive Air Shower 
Array (hereafter Yakutsk), which yields the strongest 
limits on photon flux and photon fraction in cosmic 
rays at energies E > 10 18 eV, E > 2 x 10 18 eV and 
E > 4 x 10 18 eV. These limits enter the region inter- 
esting both for highest-energy astrophysics and tests of 
extragalactic backgrounds as well as for searches of new 
physics. 

2. The key idea of the method is the event-by- 
event comparison of observed muon densities in air show- 
ers with those in simulated gamma-ray induced showers 
which have the same scintillator energy deposit and the 
same arrival direction as the observed ones. The method 
is described in detail in Ref. [30]; it has been previously 
applied to Yakutsk and AGASA muon data at highest 
energies 



3ll . |32| . One of the advantages of the method 



is its independence both from the energy-reconstruction 
procedure used by the experiment and from the Monte- 
Carlo simulation of hadronic air showers: we use simu- 
lated gamma-ray induced showers which are mostly elec- 
tromagnetic and are therefore well understood and we 



select the simulated showers by the observable scintilla- 
tor signal and not by the energy (effectively estimating 
the energy of each event in the assumption of the photon 
primary) . 

The Yakutsk extensive-air-shower array is equipped 
with five muon detectors of 20 m 2 area each with thresh- 
old energy 1 GeV for vertical muons. At present, it is 
the only installation in the world which is equipped with 
muon detectors and is capable of studying cosmic rays 
with energies above 10 18 eV. 

For the present study, we use the sample of events sat- 
isfying the following criteria: 

1) the event passed the selection cuts for the spectrum 
reconstruction; 

2) the reconstructed core location is inside the array 
boundary; 

3) the zenith angle < 45°; 

4) the reconstructed energy [46]] E rec > 10 18 eV; 

5) the reconstructed shower axis is within 300 m from 
an operating muon detector. 

The data set contains 1647 events and corresponds to the 
effective exposure of 7.4 x 10 8 km 2 • s • sr. 

By making use of the empirical muon lateral distri- 
bution function (33|, we calculated, for each event, the 
muon density at 300 m from the shower axis, ^(300), 
which we use as the composition estimator. Individual 
detector readings were evaluated from the raw data rean- 
alyzed for this study. Statistical errors of these detector 
readings were estimated on the case-by-case basis (de- 
tails of the reanalysis of the Yakutsk muon data will be 
presented elsewhere). The dominant contribution to the 
statistical error of ^(300) comes from the uncertainty in 
the determination of the shower axis (for which we use 
the geometric reconstruction from the main scintillator 
array). The overall uncertainty of p M (300) varies from 
~ 15% to ~ 40% for individual events. 

We apply the event-by-event analysis following 
Ref. [13] and estimate, for each event, the probability 
that it has been initiated by a primary photon with en- 
ergy in the range under study. To this end, we use a 
library of photon-induced showers with different energies 
and arrival directions, of which we select those with the 
same scintillator signal and zenith angle as the observed 
event, up to reconstruction errors (a detailed descrip- 
tion of the method is presented in Ref. [3(|). Since all 
events in the sample have reconstructed energies below 
10 19 eV, we do not expect azimuthal-angle dependence 
of the shower properties due to geomagnetic cascading; 
therefore we require consistency between the arrival di- 
rections of the observed and artificial showers in zenith 
angle only. To obtain the limit on the flux of primary 
photons, we slightly modified the technical part of the 
procedure of Ref. [30( . Let F 7 be the integral flux of pri- 
mary photons over a given energy range. Then we expect 
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to detect 



n(F 7 ) = F 7 A(1 - A) 



photon events on average, where A is the exposure of 
the experiment for a given dataset and A is the fraction 
of "lost" photons. Let V(n) be the probability to have 
n photons in a dataset (calculated from data following 
Ref. [30(). To constrain the flux at the confidence 
level £ one requires 

£)P(n)W(n,n(F 7 )) <l-£, 

n 

where W(n, n) is the Poisson probability to observe n 
particles for the average n. 

To simulate the shower library, we used COR- 

SIKA 6.611 [H with FLUKA 2006.3 \j$ as a low-energy 
hadronic interaction model and EPOS 1.61 [36| as a high- 
energy model. This choice of the hadronic models cor- 
responds to the highest muon density and therefore pro- 
vides a conservative limit on the gamma-ray flux; how- 
ever we checked that the difference in expected muon 
density between EPOS 1.61 and QGSJET II is neg- 
ligible for photon showers. We used thinning (10 -5 ) with 
weight limitations [HI to save computational time. The 
response of the scintillators was simulated with GEANT 
in Ref. (j^. 

Below, we present limits on the fraction of gamma rays 
and on the absolute gamma-ray flux. The flux limits do 
not depend on the choice of hadronic interaction model 
used in simulations, nor on the energy reconstruction 
used in the experiment; the only assumption is that elec- 
tromagnetic showers are simulated correctly. The frac- 
tion limits depend on the energy scale assumed for non- 
photon primaries. 

3. The upper limits on the observed flux and fraction 
of primary gamma rays are summarized in Table HI We 
compare the limits with those from previous works in 
Fig. [T] (for the gamma-ray fraction) and Fig. [2] (for the 
gamma-ray flux). 

The sensitivity of plastic scintillators to electromag- 
netic showers, strong discriminating power of large-area 
muon detectors, 25-year exposure and a sophisticated 
analysis led up to the most stringent limits on the pri- 
mary photon flux at energies above 10 18 eV and 2 x 
10 18 eV. These limits challenge previously allowed new- 
physics models and start to fill the gap between limits on 
the diffuse gamma-ray flux at ~10 16 eV and ~10 19 eV. 
The flux limits do not depend on the energy reconstruc- 
tion used by the experiment (a reconstruction in assump- 
tion of primary photons is used), nor on the simulations 
of hadronic showers. The fraction limits also use the en- 
ergy estimation in assumption of primary photons and 
also do not rely on simulation of hadronic showers; how- 
ever they depend on the assumed energy estimation of 
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0.004 
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0.006 


2 x 10 18 


0.13 


0.008 


0.005 
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4 x 10 18 


0.13 


0.041 


0.022 
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TABLE I: Upper limits (95% C.L.) on the integral flux F 7 
of photons with E > E m i n and on the fraction e 7 of pho- 
tons in the total integral flux of cosmic rays with E > -Emin. 
The flux limits (col. (2)) do not depend on the energy recon- 
struction procedure; the fraction limits are given for the as- 
sumption of correct energy reconstruction for non-photon pri- 
maries (col. (3)) and for the supposed overall shifts of ±30% 
for non-photon primaries (cols. (4), (5)), which correspond to 
the window of the systematic error [401 ] - 
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FIG. 1: Limits (95% CL) on the fraction of primary gamma 
rays in the integral flux of cosmic rays with E > i? m m from: 
this work (large Y); hybrid events of the Pierre Auger Ob- 
servatory (PAO-H) [4l[; the surface detector of the Pierre 
Auger Observatory (PAO-SD) El; Yakutsk (small Y) JH; 
reanalises of the AGASA (A H) G j and AGASA and Yakutsk 
(AY) [H data; AGASA (A) g|7nd Haverah Park (HP) g|. 



non-photon primary particles. This dependence is weak 
in the high-statistics regime, cf. Table [IJ 
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FIG. 2: Limits (95% CL) on the integral flux of primary 
gamma rays with E > Emin from: this work (Y); the surface 
detector of the Pierre Auger Observatory (PAO-SD) [i^] and 
AGASA (A; assume mixed proton-gamma composition) [44| . 
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